Papers by Ananya B. Sai
Language Models can Subtly Deceive Without Lying: A Case Study on Strategic Phrasing in Legislation (2025.acl-long)
Copied to clipboard
Atharvan Dogra, Krishna Pillutla, Ameet Deshpande, Ananya B. Sai, John J Nay, Tanmay Rajpurohit, Ashwin Kalyan, Balaraman Ravindran
| Challenge: | blatant lying or unintentional hallucination are common in large language models. |
| Approach: | They build a testbed mimicking a legislative environment where a corporate lobbyist module is proposing amendments to bills that benefit a specific company while evading identification by strong LLM detectors. |
| Outcome: | The proposed model can be used to detect deception in legislative environments and to optimize its phrasing to avoid detection by strong detectors. |
A Tutorial on Evaluation Metrics used in Natural Language Generation (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial presents the evolution of automatic evaluation metrics to their current state along with emerging trends in this field. |
| Approach: | This tutorial presents the evolution of automatic evaluation metrics to their current state . it aims to assess the extent of scientific progress made and identify areas/components that need improvement . |
| Outcome: | This tutorial presents the evolution of automatic evaluation metrics to their current state along with emerging trends in this field. |
Bi-Phone: Modeling Inter Language Phonetic Influences in Text (2023.acl-long)
Copied to clipboard
Abhirut Gupta, Ananya B. Sai, Richard Sproat, Yuri Vasilevski, James Ren, Ambarish Jash, Sukhdeep Sodhi, Aravindan Raghuveer
| Challenge: | Increasingly, people are forced to use the Web in languages they have low literacy in due to technology asymmetries. |
| Approach: | They propose a method to mine phoneme confusions for pairs of L1 and L2 and plug them into a generative model for synthetically producing corrupted L2 text. |
| Outcome: | The proposed method corrupts the popular language understanding benchmark SuperGLUE and improves performance. |
AnalystBench: Benchmarking professional long-form report generation with web-mined multimodal tasks (2026.findings-acl)
Copied to clipboard
Chau Minh Pham, Zichao Wang, Puneet Mathur, Alexa Siu, Akriti Jain, Aparna Garimella, Ananya B. Sai, Nedim Lipka, Mohit Iyyer, Varun Manjunatha
| Challenge: | Existing benchmarks decompose the end-to-end professional report generation into individual components. |
| Approach: | They propose a benchmarking tool that evaluates 20 real-world professional report generation tasks grounded in multimodal document collections. |
| Outcome: | The proposed model outperforms closed-source models on executive summarization tasks but drops significantly on long-horizon synthesis tasks. |
Improving Dialog Evaluation with a Multi-reference Adversarial Dataset and Large Scale Pretraining (2020.tacl-1)
Copied to clipboard
| Challenge: | Existing models for dialog evaluation are trained using a single relevant response and multiple random negatives. |
| Approach: | They propose a dataset to test whether model-based dialog evaluation metrics can be used to train models . they propose n-gram based metrics and embedding based ones to be used for model-driven evaluation . |
| Outcome: | The proposed model outperforms existing models on a reddit dataset on relevant responses and adversarial responses. |
Perturbation CheckLists for Evaluating NLG Evaluation Metrics (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation metrics for natural language generation are inadequate . existing metrics are not robust against simple perturbations and disagree with scores assigned by humans to perturbed output. |
| Approach: | They propose to propose checks which perturb the output and target a specific criteria and then use them to refine their evaluation. |
| Outcome: | The proposed templates show that existing evaluation metrics are not robust against simple perturbations and disagree with human scores on the perturbed output. |